-
Notifications
You must be signed in to change notification settings - Fork 38.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backport smoke/conformance test harness changes to release-1.1 from master #18686
Backport smoke/conformance test harness changes to release-1.1 from master #18686
Conversation
- Skip [Skipped] tests when testing for conformance kubernetes#17180 - Add cluster/test-conformance.sh script kubernetes#17997 - Support passing args to cluster/test-smoke.sh kubernetes#15404 - Support annotated tags in conformance test version detection kubernetes#17839
Labelling this PR as size/L |
Mesosphere CI pull/18686/head: master with CONFORMANCE_BRANCH=pull/18686/head: |
GCE e2e build/test failed for commit ffa5655. |
Conformance Results: pull/18686/head:
master with CONFORMANCE_BRANCH=pull/18686/head:
SchedulerPredicates test is fixed by #18348 (pending merge). The other failures need inspection, but are probably related to code or test changes, not the test harness. |
@k8s-bot e2e test this |
@kubernetes/goog-testing Can one of you guys look this over and give it a thumb? It's a manual backport because #17511 was already half backported and cherry-picking 6.5 PRs sounded like a PITA. No tests were changed, just harness scripts. Full e2e shouldn't be affected/required, but I kicked it off for completeness. |
GCE e2e build/test failed for commit ffa5655. |
Ran master with CONFORMANCE_BRANCH=pull/18686/head again. Looks like 2/3 of the errors were flakes. This time, only the expected SchedulerPredicates test failed (since then, #18348 has been merged).
|
A rerun of the conformance tests on this PR shows 2/3 failures were flakes, and 2 more flakes popped up. SchedulerPredicates is the only consistently failing test. We'll likely cherry-pick #18348
|
From Google's Jenkins, here's the e2e failure. Either it's a flake or release-1.1 doesn't pass conformance. Will re-run to check.
|
@k8s-bot e2e test this |
GCE e2e build/test failed for commit ffa5655. |
Jenkins GCE seems to be hosed.
|
Ran master with CONFORMANCE_BRANCH=pull/18686/head again, now that #18348 is merged. 100% conformance!
|
@k8s-bot e2e test this |
GCE e2e build/test failed for commit ffa5655. |
Same problem. It appears google's quota is filled, and I can't get ahold of an on-call to fix it. @kubernetes/goog-testing You guys have any control over this?
|
@k8s-bot e2e test this |
GCE e2e test build/test passed for commit ffa5655. |
Sorry for delay, I've been out of the office. @ixdy, can you take a look or find an owner? |
I'll have a look. |
LGTM. |
@k8s-bot test this Tests are more than 48 hours old. Re-running tests. |
GCE e2e test build/test passed for commit ffa5655. |
@k8s-bot test this [submit-queue is verifying that this PR is safe to merge] |
GCE e2e test build/test passed for commit ffa5655. |
Automatic merge from submit-queue |
Auto commit by PR queue bot
…-ci-backport Auto commit by PR queue bot
…-ci-backport Auto commit by PR queue bot
The following changes (hopefully) make it feasible to run release-1.1 branch conformance tests on master code in mesos environments. This should also fix mesos smoke tests on cherry-pick PRs to release-1.1.
(This only affects smoke/conformance tests. So e2e tests are not required.)